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Abstract 

This experimental design study examined the effects of viewing captioned instructional videos on EFL learners’ 
content comprehension, vocabulary acquisition and language proficiency. It also examined the participants’ 
perception of viewing the captioned instructional videos. The 92 EFL students in two classes, who were 
undertaking the Tape and Video Interpretation course, participated in this study. The randomly assigned 
experimental class viewed 30 episodes of captioned Connect with English and the control class viewed the same 
episodes without captions. Adopting the quantitative approach, a Michigan English Test, Content-Specific Tests 
and a questionnaire were administered to examine the participants’ content comprehension, vocabulary 
acquisition and language proficiency development as well as the experimental group’s perception towards 
viewing captioned instructional videos. Although, both groups recorded gains, the findings were in favor of the 
use of captioned instructional videos. The results showed that the effects of viewing captioned instructional 
videos are greater on vocabulary acquisition and language proficiency development than on content 
comprehension. The participants’ perceptions of the use of captioned instructional video were consistent with the 
results. They felt that it enhanced their language learning, but did not affect their comprehension of the movie 
and that captions were not a form of distraction. Pedagogical implications for EFL instructions, especially where 
multimedia technology tools may be limited is that, captioned instructional videos can be deemed as a promising 
media to enhance language learning. 

Keywords: captioned instructional video, content comprehension, vocabulary acquisition, English Language 
proficiency development, EFL learners 

1. Introduction 

English Language is neither an official language in Iran nor considered a fundamentalskill to be taught in schools 
in Iran. With the Iranian government’s move to support the national language, exposure to the language beyond 
the classroom is also limited and the public media is urged to broadcast translated versions of foreign movies. As 
Eslami-Rasekh and Valizadeh (2004) explained, only limited English programs are broadcasted on TV, radio or 
any other forms of media. In addition, albeit advancements in multimedia technologies, most language 
laboratories at the universities are equipped with conventional teaching and learning tools, such as radios, VCRs 
and headphones. If internet is available, its access is mostly limited and/or its speed is rather slow, which 
minimizes its useand hence, videos are commonly used as a teaching and learning tool. However, its potential to 
develop language proficiency is not fully harnessed. Baltova (1994, p. 33) explained that captioned films 
“provide simultaneous exposure to spoken language, printed text and visual information, all conveying the same 
message, and so promote content and vocabulary learning even in relatively inexperienced learners”. In the same 
vein, Koskinen et al. (1993, p. 36) asserted that captioned video, which presents the combination of the video 
actions with spoken dialogues and printed words, is a powerful tool and a “promising approach for improving 
students’ reading comprehension, vocabulary and motivation”. Despite favourable findings on the effects of 
captioned videos (Canning-Wilson, 2000; Hinkin, 2009; Winke et al., 2010; Hwang & Huang, 2011), both 
language learners and teachers appeared apprehensive. Danan (2004, p. 67) highlighted that language learners 
were often initially annoyed when they were first exposed to captioned programs. Language teachers too were 
reluctant and viewed captions as distracting. They believed that captions encouraged learners to “rely mainly on 
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the written texts, taking attention away from the actual spoken language, and fostered a form of laziness 
bordering on cheating”. Taylor (2005) concurred that learners felt that captions were distracting. Similar 
apprehension was also found among the Iranian language instructors, who were interviewed prior to the 
commencement of the study. Most of them were not in favour of the use of captions and therefore, not 
surprisingly, made no particularly attempt to switch the captions on when the instructional videos were screened. 
They were asking students to understand the conversations in the target language while they viewed the videos 
without captions. 

1.1 Literature Review 

With the incorporation of multimedia technology in teaching and learning, learners are exposed to different stimuli 
through various channels, such as verbal, visual, auditory, physical etc. Multiple-channel theory, which involves at 
least two input information channels, propounds that comprehension increases as learners interact with any 
combination of the different available sensory channels (Hsia, 1971). Paivio (1971), in his proposed dual coding 
theory, highlighted that information can be simultaneously represented by pictures as well as words, and these two 
information sources activate two coding systems: visual and verbal codes that are functionally autonomous and 
interconnected. Studies (Graza, 1991; Zanon, 2006; Etemadi, 2012; Winke et al., 2010) in the use to different 
channel sources have found that it is more probable to activate both coding systems in processing information than 
words or pictures alone. Koskinen et al. (1993, p. 39) highlighted that “captioned television provides a 
presentation of information that included opportunities to view the video action, hear the spoken word, and see the 
printed text”. Zanon (2006, p. 43) elaborated that “a subtitled video provides a triple connection between image, 
sound in one language and text ... this type of connection generally encourages strong associations for retention 
and language ... their combination here is necessarily very powerful”. Baggett and Ehrenfeucht (1983) asserted 
that when students are presented with both visual and verbal/auditory input, information encoding in one medium 
does not hamper the other. 

When captioned technology first appeared in the 1990s, educators and researchers (Holobow et al., 1984; 
Koskinen et al., 1993; Vanderplank, 1993; Koskinen et al., 1996) quickly saw the potentials of captioned videos, 
which allow for the audio-visual-print interactions to enhance multi-sensory processing, i.e. to enable students’ to 
process language effectively in both the visual and printed forms. Captioned TV programs and videos were 
reported to aid the hearing impaired (Nugent, 1983), the disabled (Wilson & Koskinen, 1986) as well as ESL/EFL 
learners (Garza, 1991; Markham & Peter, 2003). Second language (L2) researchers highlighted the significance of 
LI and L2 captions in developing specific language skills: listening (Garza, 1991; Markham, 1989; Huang & 
Eskey, 2000; Markham & Peter, 2003; Hayati & Mohmedi, 2009), content comprehension (Holobow et al., 1984; 
Grignon et al., 2005; Grgurvoic & Hegelheimer, 2007; Etemadi, 2012), reading comprehension (Garza, 1991; 
Koskinen et al., 1993; Markham & Peter, 2003; Hwang & Huang, 2011) and vocabulary acquisition (Huffman, 
1986; Neuman & Koskinen, 1992; Koskinen et al. 1996; Koolstra & Beentjes, 1999; Markham & Peter, 2003; 
Yukel & Tanriverdi, 2009; Stewart & Pertusa, 2004; Etemadi, 2012) and language learning (Borras & Lafayette, 
1994; Zanon, 2006). We will present brief descriptions of some of the related research below. 

Yukel and Tanriverdi’s (2009) examined the effects of captioned movies on incidental vocabulary learning, on 
120 intermediate universitylevel EFL Turkish students, who viewed a closed-captioned episode of a TV series, 
Seinfeld. Adopting the pre-test-post-test experimental and control group design, they were randomly assigned to 
two groups: Group A watched the captioned movie clip and Group B watched the movie clip without caption. 
The students watched the first 9 minutes and 14 seconds of the movie clip twice. A 20 items Vocabulary 
Knowledge Scales (VKS) was administered twice; prior to (two weeks before treatment) and after the treatment 
(one month after treatment). Although Group A performing slightly better than Group B, the improvement was 
not statistically significant. Winke et al. (2010) examined the effects of captioning during video-based listening 
activities, which involved 150 second and fourth year Arabic, Chinese, Spanish and Russian EFL learners in a 
U.S university. They watched a series of three, approximately 3-5 minutes English language documentaries 
videos about animals, twice, i.e. once with captions and the other without captions. The screening was alternately 
presented, first with captions and the second without captions to all the cohorts, except the Spanish cohort (as 
there were two extra groups). One Spanish cohort viewed the video without captions and the other with captions. 
At the end of the second viewing, the students sat for two tests: i.e. content comprehension and vocabulary tests. 
Viewing captioned videos with aural input was found to be more effective as significantly higher scores were 
recorded on both tests. Winke et al. (2010) highlighted that captioned, rather than non-captioned videos aid novel 
vocabulary acquisition and overall comprehension of the videos, which is facilitated by the use of different 
modalities. They also found that the order of viewing affects recognition of vocabulary; the students who viewed 
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the captioned videos first were better able to aurally recognize novel vocabulary than those who were presented 
with captioned videos in the second viewing. 

More pertinent to this study, in terms of the language focus (comprehension/vocabulary), target group (EFL 
undergraduates), context of study (Iran) and instruments (standardized battery/CST/researcher designed tests) 
used to gather data are studies by Hwang and Huang (2011), Zarei (2009) and Etemadi (2012). Hwang and Huang 
(2011) examined if captioned instructional videos improve or impede Taiwanese EFL learners’ reading 
comprehension. 80 college freshmen were randomly assigned to the experimental and control groups. The 
experimental group viewed the captioned videos and the control group viewed the non-captioned videos to for one 
hour every two weeks over 10 weeks (a period of 5 weeks). Two instruments were used to gather data: (i) a Content 
Comprehension Test (CST), which was designed by the researchers to assess the participants’ comprehension 
(listening and content) and vocabulary acquisition of only the 50 th episode, (ii) General English Proficiency (GET), 
a standardized test that the Taiwan Ministry of Education designed to assess students’ overall language proficiency, 
in terms of listening and reading skills. The descriptive statistics of the reading comprehension sub-test of the GET 
showed that the experimental group out-performed the control group, however, the statistical analysis showed no 
significant differences between the mean reading scores. 

Zarei’s (2009) conducted a study to examine the effects of bimodal, standard and reversed subtitling on L2 
learners’ vocabulary recognition and recall. The participants of this study were made up of 92 BA level Iranian 
students (3 groups), who viewed nine episodes (30 minutes per episode) of a British TV comedy at the last half 
an hour of each class session (which was an additional task in the regular class activities and materials). All 
groups viewed the same film. The first group viewed the film with both the English soundtrack and subtitles, the 
second group viewed the film with English soundtrack and Persian subtitles, and the third group viewed the film 
with Persian soundtrack and English subtitles. They were also required to view the films as homework and short 
content comprehension quizzes were carried out to ensure they viewed the films at home. Two vocabulary tests, 
which the research designed, were administered as post-tests. The first vocabulary test, (40 multiple-choice 
questions), was administered at the end of the study, to examine the participants’ comprehension of the words, 
that appeared in the film. The second vocabulary test (40 fill-in-the-blank questions) was administered a week 
after the study to measure the participants’ recall of the vocabulary. A 40-item multiple-choice vocabulary 
sub-section of the Michigan test was also administered to gauge the homogeneity of the participants, before the 
study as well as a post-test to valid the post-tests that were administered at the end of the study. This study found 
no significant difference between the bimodal and standard subtitling groups for vocabulary recognition, 
however, both groups performed significantly better than the reversed subtitling group. The results of vocabulary 
recall, on the other hand, showed that bimodal subtitling was significantly more effective than standard subtitling, 
which in turn was significantly more effective than reversed subtitling. 

Etemadi’s (2012) one-shot case study examined the effects of bimodal captions of English movies on content 
comprehension and vocabulary recognition. The 44 senior Iranian undergraduates in her study watched two BBC 
documentaries, with and without English subtitles, in one session. The screening of the two documentaries was 
reversed to counteract the order effects. Class 1 viewed Dangerous knowledge with captions, followed by Where’s 
my robot? Without captions. Class 2 viewed the same documentaries in reversed order. Immediately after viewing 
each of the documentaries, the participants answered a set of 20 multiple choice questions (10 questions each on 
vocabulary and content comprehension) per documentary. The researcher designed the two sets of tests (a total of 
40 multiple choice questions: 20 questions each on vocabulary and content comprehension). The /-test results 
showed positive effects of the bimodal subtitling on content comprehension, but not on vocabulary recognition. 

Koskinen et al. (1993, p. 41) asserted, “The instructional potential of captioned video is enormous”. The 
plausible effects of captioned instructional video in enhancing Iranian EFL learners’ integrated language skills, 
i.e. comprehension vocabulary acquisition and language proficiency was one of the motivations to carry out this 
study. As mentioned earlier, most of the language instructors who were interviewed prior to this study appeared 
apprehensive of the effectiveness of captions. Secondly, based on our google search, we found limited studies 
werecarried out in Iran and except for two studies (Etemadi, 2012; Zarei & Rashvand, 2011), the rest of the 
studies appeared to focus on the effects of captions on isolated language skills: listening (Latifi et al., 2011; 
Hayati & Mohmedi, 2009; Ghasemboland & Nafissi, 2012) and vocabulary (Zarei, 2009; Taghavi et al., 2012). 
The two studies that examined integrated language skills are Zarei and Rashvand’s (2011) study that focused on 
vocabulary and production, and Etemadi’s (2012) study on content comprehension and vocabulary. Thirdly, 
there also appears to be limited studies that examine the effectiveness of captioned instructional video on 
integrated language skills. Most studies, as mentioned earlier, appeared to focus on isolated language 
components, i.e. listening (Kusumarasdyati, 2005; Hayati & Mohmedi, 2009), reading and/or content 
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comprehension, language learning (Borros & Lafayette, 1994; Grignon et al., 2005; Zanon, 2006) and mostly on 
vocabulary (Garza, 1991; Koolstra & Beentjes, 1999; Birds & Willians, 2002; Stewart & Pertusa, 2004; Hinkin, 
2009). 

Thus, this study had set out to, firstly, provide evidence for the use of captions in a Tape and Video 
Interpretation course in addressing the language instructors’ apprehension, secondly, to fill in the research gap by 
extending previous studies and examining three integrated components, i.e. comprehension (content/reading), 
vocabulary acquisition and language proficiency development as well as examining the students’ perceptions of 
viewing captioned instructional videos, and finally, to contribute to the body of literature, particularly in the EFL 
context, where the possibility of using advanced multimedia technologies in classrooms/language laboratories 
may be limited to the use of only VCRs. 

1.2 Research Questions 

This study addressed the following research questions: 

1) Does the presence of English language captions in instructional videos improve EFL learners’ 

a) Content comprehension? 

b) Vocabulary acquisition? 

c) Language proficiency? 

2) What are the EFL learners’ perceptions of captioned instructional videos? 

2. Methodology 

2.1 Participants 

Participants of this study were made up of all 92 (33% male and 76% female) EFL students undertaking the Tape 
and Video Interpretation course at Islamic Azad University of Mashhad, Iran. Their ages ranged from 19 to 24 
years. Since two classes were held, one was randomly set as the experimental (45 students) and the other as the 
control (47 students) groups. To gauge the participants’ level of proficiency before the treatment, a Michigan 
English (2009) Test (MET) was administered and the results of the pre-MET is presented in Table 1. 


Table 1. Pre-MET 


Groups 

N 

Mean Scores 

Std. Deviation 

Std. Error Mean 

Experimental 

45 

67.42 

12.41 

1.85 

Control 

47 

63.10 

12.84 

1.87 


As shown in Table 1, the descriptive statistics of the results of the pre-MET confirmed that the participants in 
both the experimental and control groups were approximately of the same level of language proficiency; they 
recorded mean scores within the 60 - 70 range, with a mean difference of < 5 (Experimental: M = 67.42; Control: 
M = 63.10), which implied that they were homogeneous and at the intermediate level. To verify if there was 
statistical difference between the groups, we ran a /-test for equality of means and the results, which are 
presented in Table 2, show that the mean difference between the two groups was not statistically significant: / 
(90) = -1.63,/? = 0.105 > .05. 
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Table 2. T-test for equality of mean for pre-MET 
Independent sample test 


Levene’s 
Test for 
Equality of 
Variances 


T-test for Equality of Mean 


F Sig. T 


95% Confidence 

Sig. Mean Std Error Interval of the 

(2-tailed) Difference Difference Difference 


Lower Upper 


Pre- 

Equal 

Variances .071 

assumed 

.791 -1.63 

90 

.105 

-4.31 

2.63 

-9.55 

.92 

MET 

Equal 









Variances 
not assumed 

-1.63 

89.99 

.105 

-4.31 

2.63 

-9.55 

.91 


Hence, it is possible to contend that any change in the experimental group’s mean scores, at the end of the 
treatment, would be deemed as the presence of English language captions in instructional videos improves EFL 
learners’ language proficiency. 

2.2 Materials 

An instructional video series, Connect with English (McGraw-Hill, 1998) was used in this study. Connect with 
English (CWE) is a soap opera about a journey of a young aspiring singer, Rebecca Casey, who leaves Boston to 
pursue her dreams in the music in California. The complete CWE series comprises 25 videos, with 50 
fifteen-minute episodes (including the Introduction and Meet with Discussion Group). Every two 15 minute 
episodes are on a specific theme. McPartland-Fairman et al. (1998) noted that the language levels of each 
episode pursue a sequence. The vocabulary in the earlier episodes is simpler compared to the later episodes, 
which is more advanced. They elaborated that the “key lines are repeated, important events are retold and idioms 
are paraphrased.. .there are constant visual clues, such as notes, documents, written signs, facial expressions, 
gestures and body language to convey meaning”. 

Upon confirming that the 92 participants have not viewed the video prior to the study, both groups were briefed 
on the story leading up to the events relating to the 18 th episode (Intermediate level). The decision to begin the 
treatment with episode 18 was based on the results of the pre-MET, which showed that the participants were at 
the intermediate level as well as the description of the levels of the episodes in the Teacher’s Guide of the 
instructional video. The experimental group watched CWE with captions and the control group watched the same 
video without captions. 

2.3 Instruments 

Adopting the inferential statistic procedure, this study gathered quantitative data, using the MET, the Content 
Specific Tests (CST) and a questionnaire to address the research questions. The MET and CST were used to 
gather data to answer the first research question ( Does the presence of English language captions in the 
instructional video improve EFL learners ’ content comprehension, vocabulary> acquisition and English language 
proficiency?). Thea questionnaire was used to gather data on to answer the second research question ( What are 
the EFL learners ’perceptions of captioned instructional video?). 

2.4 Michigan English Test (MET) 

Briefly, the MET is for adults and adolescents at or above a secondary level of education non-native speakers of 
English, who will need to use English for academic purposes at the college and university level (University of 
Michigan, 2010). It is used to evaluate English language proficiency in different contexts, including education. It 
constitutes five sub-sections: reading comprehension (three reading passages, 25 multiple choice questions to be 
completed in 35 minutes), listening comprehension (25 multiple choice questions, to be completed in 20 
minutes), vocabulary and grammar (25 multiple choice items each, to be completed in 15 minutes for each 
component), and writing (this sub-section was not administered for two reasons, firstly, as the manual specifies, 
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it is recommended to be administered when trained raters are available to assess the scripts and secondly, it was 
not the focus of this study). The scoring of MET followed the manual guidelines, i.e. each item was scored “1” 
for a correct answer and “0” for an incorrect answer. Since the MET comprised 100 multiple choice questions, 
the possible maximum score was 100 marks. Hence, we deemed that the total scores attained reflected the 
improvement in the participants’ language proficiency. 

The Michigan English 2009 Test Administration Report published by the University of Michigan’s English 
Language Institute Testing and Certification Division reported very high reliability, i.e. between 0.90 and 0.93 
(University of Michigan, 2010) for all sub-sections of the MET. Winster et al. (2009) too reported a reliability of 
r = 0.75. A sample of the questions is attached as Appendix A. 

2.5 Content Specific Tests (CST) 

The CST, which the Testing Centre of the English Department of Islamic Azad University developed, was 
designed based on the major concepts and information presented in the video. The tests were adopted to examine 
the participants’ content comprehension as well as vocabulary acquisition. Each content specific test comprised 
eight questions (four questions on content comprehension and another four questions on vocabulary sections) on 
one theme, the complete set of CST (15 sets of tests) was made up of 120 multiple choice questions (60 
questions for each of the content comprehension and vocabulary sections). The duration for each test was eight 
minutes (4 minutes for each section). Similar to the MET, correct responses were awarded “1” point and 
incorrect responses were scored as zero, and the maximum possible score was 120 marks. The total scores for 
the 15 tests were tallied to record the participants’ final scores. A sample of the CST question is attached as 
Appendix B. 

To ensure content validity, three experienced (more than 10 years of teaching experience) EFL instructors, who 
viewed the video prior to the commencement of the study, reviewed the test items. A five-point Likert-scale was 
adopted: (1) Very Appropriate, (2) Appropriate, (3) Fair, (4) Inappropriate, and (5) Very Inappropriate. Items 
rated below the scale of 2 were revised as proposed by the instructors and a coordination meeting was held with 
the instructors to verify the 120 multiple choice questions. 

2.6 Questionnaire 

The researchers designed a questionnaire to examine the experimental group’s perceptions of captioned 
instructional videos as well as to triangulate data. It comprised six statements: four statements that required 
responses of strongly disagree to strongly agree (using the 5 Likert scale) and two statements, which required a 
“yes - no” response (See Appendix C). Of the six statements, two statements (statement 4 in section A: “When / 
read the words in the captions, I miss the story’ line.” and statement 1 in section B: “7 have a better 
understanding of the movie when I read the subtitles.”) were aimed at collecting data on the participants’ 
perception of the effects of viewing captioned instructional movie on content comprehension. The remaining 
three statements in section A {My Knowledge of new words increased with the help of captions, I understand the 
meaning of words better with the help of the actions in the instructional videos, I tend to remember the new 
words that I come across in the captions) were aimed at gathering feedback on the participants’ perception of the 
effects of viewing captioned instructional video on vocabulary acquisition. Statement 2 in section B (/ learn 
English better with subtitled instructional videos) was aimed at eliciting their perception on whether captioned 
instructional video is an effective tool for learning English. The same three EFL instructors, who validated the 
CST, verified the face validity of the questionnaire. 

2 .7 Data Collection 


Table 3. Data collection schedule 



Data 

Collection 




Data Collection 

Weeks 


1 

2 3 4 

5 

n.-f A/TT7T 

a Class Sessions 

Pre-MET 

1 2 3 

4 5 6 7 8 9 10 11 

12 13 14 

^ rOSt-Mbl 

Questionnaire 

Episode Numbers 


18-23 

24-29 30-35 36-41 

42-47 



“Content-Specific Tests were administered at the end of each class session 
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Table 3 displays the schedule for treatment and the administration of the instalments. It was conducted over a 
five week period, with three sessions per week, which totaled 15 sessions. The participants viewed two episodes 
for 90 minutes per class session, beginning with episode 18 and ending with episode 47; they viewed a total of 
30 episodes (15 themes). 

In week 1, the participants viewed episodes 18 to 23 (2 episodes per week), i.e. 18-19 in session 1, 20-21 in 
session 2 and 22-23 in session 3. In week 2, another three class sessions (4 to 6) were held and the participants 
viewed another three episodes (24 to 29). This goes on for weeks 3 (episode 30-35), 4 (episode (36-41) and 5 
(episode 42-47). The MET was administered twice, i.e. as a pre-test and post-test to gauge the participants’ 
language proficiency before and after the treatment respectively. Since one CST was administered at the end of 
each class session, the participants sat for 15 content specific tests, which were aimed at examining the 
participants’ content comprehension and vocabulary acquisition. The third instrument was administered at the 
end of final treatment to elicit their perception of the use of the captioned instructional video. 

3. Results 

The results of the data analysis of the MET and CST, using the SPSS software version 16, are presented as 
descriptive statistics. The /-test equality of means, which was conducted to verify the variance of the mean scores 
of both the results are also presented to support the findings. The level of significance was set at 0.05. Responses 
to the questionnaire are tabulated as percentages. 

The results presented below are in response to the research questions, which are reiterated here for ease of 
reference: 

1) Does the presence of English language captions in instructional video improve EFL learners’ 

a) Content comprehension? 

b) Vocabulary acquisition? 

c) Language proficiency? 

2) What are the EFL learners’ perceptions of captioned instructional videos? 

3.1 Content Comprehension and Vocabulary / Acquisition 

In addressing parts (a) and (b) of the first research question, this section presents the results of the CST on 
content comprehension and vocabulary acquisition. Table 4 presents the descriptive statistics of the CST 
computed scores for both the sub-sections. 


Table 4. Descriptive statistics of sub-sections of Content Specific Tests 



Mean Scores (Standard Deviations) 

Std. Error Mean 


Groups 

Content 

Comprehension 

Vocabulary 

Content 

Comprehension 

Vocabulary 

Experimental 

46.00 (8.09) 

46.20 (9.38) 

1.207 

1.39 

Control 

44.64 (8.58) 

31.79 (13.98) 

1.253 

2.04 


The results clearly show that the experimental group performed better than the control group in both sub-sections. 
The experimental group recorded slightly higher mean scores, i.e. > 46 (for both sub-sections) than the control 
group’s mean scores of < 45. The experimental group recorded mean scores that were almost similar, with a .20 
variant for both sub-sections (Content Comprehension, M = 46.00; Vocabulary, M = 46.20). In contrast, the 
control group recorded more varied mean scores; they performed better in content comprehension (M = 44.64) 
than in vocabulary (M = 31.79). A similar pattern is seen in the standard deviations of both the groups. A more 
varied standard deviation was found for the control group in both sub-sections (Content Comprehension, SD = 
8.58; Vocabulary, SD = 13.98) than the experimental group (Content Comprehension, SD = 8.09; Vocabulary, 
SD = 9.38). In comparing the performance in each of the sub-sections separately, the difference between the 
means scores and standard deviations for the content comprehension section was not obvious, i.e. MD = 1.36 
points (Experimental, M= 46.00; Control, M= 44.64) and a difference of .49 in SD (Experimental, SD = 8.09; 
Control, SD = 8.58) respectively. On the contrary, there was a wider mean difference of 14.41, in favour of the 
experiment group for the vocabulary section (Experimental, M = 46.20; Control, M = 31.79, but the variance in 
scores within the experimental group (SD = 9.38) was smaller compared to the control group (SD = 13.98). To 


7 




www.ccsenet.org/elt 


English Language Teaching 


Vol. 7, No. 5; 2014 


verify the results, we ran the /-test to examine if there was any statistical difference between the groups. The 
results showed that there was no significant difference for content comprehension (/ (90) = -.73, p = .436), 
indicating that the experimental group did not out-perform the control group. However, the wider difference in 
mean scores between the groups for vocabulary was significant, / (90) = -5.78, p = .000 (See Table 5). 


Table 5. T-tcst for equality of means for Content Specific Tests 




T 

df 

Sig. 

(2-tailed) 

Mean 

Difference 

Std. Error 
Difference 

95% Confidence 
Interval of the 
Difference 








Upper 

Lower 


Equal Variances 
assumed 

Equal Variances 
not assumed 

-.728 

90 

.436 

-1.362 

1.742 

-4.822 

2.099 

Content 

-.783 

89.980 

.436 

-1.362 

1.740 

-4.818 

2.094 


Equal Variances 
assumed 

Equal Variances 
not assumed 

-5.779 

90 

.000 

-14.413 

2.494 

-19.367 

-9.458 

Vocabulary 

-5.827 

80.774 

.000 

-14.413 

2.473 

-19.334 

-9.491 


3.2 Language Proficiency 

As mentioned earlier, in addressing the research question “Does the presence of English language captions in 
instructional videos improve EFL learners ’ language proficiency? ” The MET was re-administrated at the end of 
the treatment to examine if there was improvement in language proficiency. The descriptive statistics of the pre- 
and post-MET are displayed in Table 6 for ease of comparison. 


Table 6. Descriptive statistics ofpre- and post-MET 


Groups 

N 

Mean Scores 

Std. Deviation 

Std. Error Mean 

Pre-MET 

Post-MET 

Pre-MET 

Post-MET 

Pre-MET 

Post-MET 

Experimental 

45 

67.42 

82.88 

12.41 

5.94 

1.85 

0.88 

Control 

47 

63.10 

68.12 

12.84 

13.06 

1.87 

1.90 


The descriptive statistics in Table 6 shows that both groups benefited from viewing the captioned instructional 
video. As mentioned earlier, before the treatment, both groups were approximately of the same level of language 
proficiency, i.e. at the intermediate level (Experimental, M = 67.42, SD = 12.41; Control, M = 63.10, SD = 
12.84), however, after the treatment, although positive effects of viewing the instructional videos were evident 
for both groups, the experimental groups’ performance was much better than the control group. The level of 
language proficiency of the experimental group improved to the advanced level, i.e. with means scores of >80 
( M = 82.88) compared to the control group, who remained at the intermediately level (< 70; M = 68.12), but 
recorded higher means scores in the post-test than the pre-test (M = 163.10). When the mean scores between the 
pre- and post-tests for each group are compared, the experimental group recorded an obvious increase of 15.46 
(pre-MET: M = 67.42; post-MET: M = 82.88) compared to the control group, who recorded an increase of only 
5.02 (pre-MET: M = 63.10; post-MET: M = 68.12). As for the variance in scores within the groups, the standard 
deviations between the groups in the post-test was smaller for the experimental group (SD = 5.94) than the 
control group (SD = 13.06). The experimental group clearly out-performed the control group, with a wider mean 
difference, i.e. MD = 14.76 (Experimental, M = 82.88; Control, M = 68.12) in the post-test. The mean difference 
in the pre-test, however, wassmaller, i.e. MD = 4.32 (Experimental, M = 67.42; Control, M = 63.1 0). Once 
again, we ran a /-test to examine if the differences in mean scores were statistically significant (See Table 8). 
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Since this study examined the effects of viewing the instructional video on content comprehension and 
vocabulary acquisition (Research Question 1), we analyzed the participants’ performance in the individual 
reading comprehension and vocabulary sub-sections of the post-MET further (See Table 7). 


Table 7. Descriptive statistics of reading comprehension and vocabulary of post-MET 


Group Statistics 







Group 

N 

Mean 

Std. Deviation 

Std. Error Mean 


Experimental 

45 

18.53 

3.49 

0.52 

Reading Comprehension 


Control 

47 

16.65 

3.75 

0.54 


Experimental 

45 

19.13 

3.18 

0.47 

Vocabulary 


Control 

47 

17.04 

3.53 

0.51 

Reading Comprehension & Experimental 

45 

37.66 

6.31 

0.94 

Vocabulary 

Control 

47 

33.70 

6.88 

1.00 


As shown in the table, there was only a slight difference in performance between the groups, in favour of the 
experimental group. Although the mean difference between the groups for the reading comprehension section 
was < 2, i.e. 1.88 (Experimental, M = 18.53; Control: M = 16.65), a slightly wider mean difference of > 2, i.e. 
2.09, in favor of the experimental group for the vocabulary sections (Experimental, M = 19.13; Control: M = 
17.04) was recorded. With the slight difference in performance in the individual sub-sections, we decided to 
analysis the results further by combining the scores of both the sub-sections to draw a more reliable conclusion. 
We found a wider difference of 3.96 (Experimental, M = 37.66; Control, M = 33.70) when the scores of both 
sub-sections were combined. Similar to the CST, we ran a/-test for the total scores attained in the post-MET, 
individual sub-sections for the reading comprehension and vocabulary sections as well as the combined 
sub-sections to examine if there were statistical differences in the mean scores. The results are presented in Table 
8 . 


Table 8. T-test for equality of mean 


Group Statistics 







Group 

N 

Mean 

Std. Deviation 

Std. Error Mean 


Experimental 

45 

18.53 

3.49 

0.52 

Reading Comprehension 


Control 

47 

16.65 

3.75 

0.54 


Experimental 

45 

19.13 

3.18 

0.47 

Vocabulary 


Control 

47 

17.04 

3.53 

0.51 

Reading Comprehension Experimental 

45 

37.66 

6.31 

0.94 

& Vocabulary 

Control 

47 

33.70 

6.88 

1.00 


The /-test results confirm that the experimental group performed significantly better than the control group in the 
overall performance of the post-MET (/ = -6.92, p = .000), in the individual sub-sections, i.e. reading 
comprehension (/ = -2.47, p = .015) vocabulary (/ = -2.97, p = .004) as well as the combined sections (/ = -2.87, 
p = .005). 

3.3 Perception of Captioned Instructional Video 

To recap, a questionnaire was administered to elicit the participants’ perception of viewing the captioned 
instructional video and to triangulate data. The descriptive analysis of the sections A and B are presented in 
Tables 9 and 10 respectively. 
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Table 9. Responses to the section A of questionnaire 

Section A 


SD* 

D* 

NS* 

A* 

SA* 

Items 

N (%) 

F* (%) 

F (%) 

F (%) 

F (%) 

F (%) 

Q1 (Knowledge of new 
words increased with the 
help of captions) 

45 (100%) 

0 (0%) 

0 (0%) 

0 (0%) 

35 (77.8%) 

10 (22.2%) 

Q2 (Understand meaning of 

words better with help of 45 (100%) 

actions in movie) 

0 (0%) 

10 (22.2%) 

9 (20.0%) 

16 (35.6%) 

10 (22.2%) 

Q3 (Tend to remember new 
words in captions) 

45 (100%) 

0 (0%) 

12 (26.7%) 

10 (22.2%) 

23 (51.1%) 

0 (0%) 

Q4 (Missed story’ line while 
reading captions) 

45 (100%) 

6 (13.3%) 

22 (48.9%) 

13 (28.9%) 

4 (8.9%) 

0 (0%) 


* SD: Strongly Disagree; D: Disagree; NS: Not Sure; A: Agree; SA: Strongly Agree; 

* F: Frequency 


At a quick glance, we can that the responses varied from strongly agreeing to strongly disagreeing to three 
statements (Statements 2, 3, 4), except statement 1 ( Knowledge of new words increased with the help of 
captions). All 45 participants affirmed (77.8% Agreed, 22.2% Strongly Agreed) that the provision of captions 
attributed to increase in vocabulary, however, the responses to statement 2: “/ understand the meaning of words 
better with the help of the actions in the videos ” were more varied. While 26 (57.8%) participants agreed (35.6% 
agreed, 22.2% strongly agreed) that actions in the video attributed to better understanding of the meaning of 
words, 10 (22.2%) participants disagreed and the remaining 9 (20%) participants were unsure, of the scaffolding 
effects of an action in the video to help decipher the meaning of the words. The third statement: “/ tend to 
remember the new words that I come across in the caption” was intended to examine how sustainable the effects 
of providing captions were in terms of remembering new words. Approximately half of the participants, i.e. 23 
(51.1%) participants agreed that they tend to remember the new words in the captions, 12 (28.3%) participants 
felt that they tend to forget the new words and 10 (22.2%) participants were unsure as to whether captions were 
helpful. In comparison, a larger proportion of the participants, i.e. 28 (62.2%) participants disagreed with 
statement 4 ( When I read the words in the captions, I miss the story line). 13 (28.9%) participants noted they 
were unsure and only 4 (8.9%) participants felt that they missed the story line. 

Responses to section B of the questionnaire, i.e. the participants’ perceptions toward the effects of captions on 
content comprehension and language learning are presented in Table 10. 


Table 10. Responses to section B of questionnaire 


Section B 


YES 

NO 

Items 

N (%) 

Frequency (%) 

Frequency (%) 

Q5 (Better understanding of the 
movie when read the captions) 

45 (100%) 

4 (8.9%) 

41 (91.1%) 

Q6 (Learn English better with 
captioned video) 

45 (100%) 

35 (77.8%) 

10 (22.2%) 


In general, the number of responses recorded for both statements in Section B appears to indicate that the 
majority of the participants felt that captions did not attribute to enhanced understanding of the content of the 
video, nevertheless, favoured the captioned instructional video as a tool to learn English. While, only a small 
proportion of participants, i.e. 4 (8.7%) perceived that they had “... a better understanding of the movie when 
[they] read the captions”, a majority of participants, i.e. 41 (91.1%) felt that captions did not enhance their 
comprehension of the movie. Conversely, a larger proportion of participants, i.e. 35 (77.8%) perceived that they 
learn English better with captioned videos ” and only 10 (21.7%) participants felt otherwise. 
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4. Discussion 

This section presents the discussion on the ftndingsin response to the research questions, which are reiterated for 
ease of reference: 

1) Does the presence of English language captions in instructional videos improve EFL learners’ 

a) Content comprehension? 

b) Vocabulary acquisition? 

c) Language proficiency? 

2) What are the EFL learners’ perceptions of captioned instructional videos? 

The discussions will begin with the effects of viewing instructional captioned videos on content comprehension 
and vocabulary acquisition, followed by language proficiency development. The participants’ perceptions 
towards instructional captioned videos are discussed simultaneously to triangulate data in support of the findings. 

4.1 Content Comprehension 

In reporting the effects of captions on content comprehension, we analyzed the results of the comprehension 
sub-section of the CST, reading comprehension section of the MET and questionnaire. The descriptive statistics 
and independent samples /-test on content comprehension (CST) clearly points to no obvious/significant 
difference in performance between the groups. This is consistent with the views of most of the participants in the 
experimental group, who felt that the printed form of the dialogues appeared to neither attribute to better 
understanding of the movie, nor affect their understanding of the storyline. A larger proportion of the 
participants felt that captions, as Koostra and Beentjes (1999) noted, are far from distracting, and this contradicts 
Taylor’s (2005) findings that it affects comprehension of the movie. A concern for lack of familiarity may lead 
to a sense of distraction, as raised by Lambert and Flolobow (1984), may not be the case in this study, since the 
participants in this study had not viewed CWE, which is set in the US, prior to the treatment. These results also 
appear to contradict Holobow et al. (1984), Grignon et al. (2005), Grgurovic and Hegelheimer (2007), and 
Etemadi’s (2012) findings that captions have a positive effect on content comprehension. The results of the 
reading comprehension section of the MET, which contradicts Hwang and Huang’s findings (2011), showed 
favourable effects of captioned instructional video. The significant difference between the groups is consistent 
with Garza’s (1991) and Koskinen et al.’s (1993) findings. 

4.2 Vocabulary’ Acquisition 

Similar to content comprehension, data on vocabulary acquisition were obtained from the three instillments. 
Interestingly, in contrast to the findings on content comprehension, but similar to reading comprehension, the 
effects of viewing captioned instructional videos on vocabulary acquisition was significant. The experimental 
group, who was able to “hear” the dialogue and “see” the printed captions simultaneously, displayed improved 
vocabulary acquisition, compared to the control group. Once again, consistency was found in the test results and 
the participants’ perception towards vocabulary acquisition. All the students participants felt that captions helped 
improve their vocabulary, which was reflected in their significant performance in the vocabulary sub-sections of 
the CST and MET. Most of the participants also felt that they understood the meanings of words better when the 
printed forms appeared concurrently with the actions in the video. The visual stimuli of the actions appeared to 
attribute to better understanding of the vocabulary. Our results, therefore, contradict Etemadi’s (2012) findings 
that bimodal captioning did not affect vocabulary acquisition. However, our results are consistent with findings 
reported by Neuman and Koskinen (1992), Koskinen et al. (1993), Zarei (2009), Yuksel and Tanriverdi (2009), 
and Winke, Gass, and Sydorenko (2010). 

In general, our findings suggest that viewing captioned instructional video has a positive effect on vocabulary 
acquisition, but not on content comprehension. This finding appears to contradict Etemadi’s (2012) result which 
suggested otherwise. Her study reported the reverse effects, i.e. positive effects of captioned movies on content 
comprehension, but not vocabulary. However, it ought to be noted that she adopted the “one-shot” case study 
and ours was an experimental study, which was conducted over a 5-week period and the participants viewed a 
series of episodes. 

4.3 Language Proficiency 

In response to the research question, “ Does the presence of English language captions in instructional videos 
improve EFL learners’ language proficiency ?” We found that viewing instructional videos, irrespective of 
captioned or non-captioned, enhances English language proficiency. It attests to what Koskinen et al. (1993) 
maintained, i.e. a promising approach for enhancing language proficiency. However, it appears to be an even 
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more promising approach, if captioned videos are used as a teaching and learning tool in an EFL context. We 
found that captioned videos appear to have a greater effect on language proficiency than non-captioned videos. 
The experimental group recorded greater improvement than the control group. Recording a greater increase in 
mean score (MD = 15.46) compared to the control group (MD = 5.02), their level of language proficiency 
improved to the advanced level. The /-test verified the significant difference in the overall performance of the 
post-MET as well as the individual and combined sections of the reading comprehension and vocabulary. The 
participants’ perception of the use of captions, i.e. that they learn English better with the aid of captions is 
reflected in the results. This improvement in English language proficiency is consistent with the findings by 
Borras and Lafayette (1994), and Zanon (2006). As Paivio (1971) highlighted, this study too found that multiple 
presentation of information in spoken, written and visual forms had contributed positively to increased 
performance and improved language proficiency than single presentation. 

5. Limitations of Study 

Several limitations are recognized and acknowledged in this paper. Among the limitations were the language 
components, choice of instruments and duration of study. Firstly, although this study examined integrated 
language skills on content comprehension, vocabulary acquisition and language proficiency, other language skills, 
such as listening, speaking, grammar and writing were not examined. Secondly, we used the MET, which is a 
standardized test battery and the CST, which were available at the university. The duration of each CST was 
limited to only eight minutes per test, i.e. one minute per question, which appears to be a speed test. Within the 
limited class time, we were unable to allocate more time. Inclusive of the questionnaire, we collected only 
quantitative data and therefore we were unable to report the participants’ feedback beyond the statements in the 
questionnaire. In addition, a delayed post-test was not administered to determine if the effects of viewing 
captioned instructional video lasted. The third limitation was the duration of study, which entailed limited episodes 
were screened, i.e. over a five week period, the participants viewed only 30 of the total 50 episodes, starting with 
episode 18 to 47. 

The results may have varied if different materials and instruments were used as well as the the time allocation for 
the CST and the duration of the study were longer. The use of other materials, such as documentaries, 
non-instructional videos, cartoons may yield a different set of results. We found that familiarity of the movie does 
not affect performance, however, if familiar movies or movies with the local context were used, the performance 
of the students may have been even better. The results may have varied if other assessment tools and standardized 
test battery were administered, or even if other types of questions, such as open-ended or structured questions were 
used, instead of multiple choice questions. The results may have also differed, if the students were given the 
opportunity to view the complete CWE series, i.e. all 50 episodes from the beginning, rather than only 30 episodes. 

6. Recommendation for Future Research 

Suggestions for future research include examining a more integrated component of the language skills, i.e. 
productive (oral/written) and receptive (grammar/reading/listening) skills. Based on our literature review, a larger 
proportion of studies appear to examine vocabulary acquisition. There appears to be no specific study that 
examined the effects of captions on grammar and there is also limited study on the oral and written productive 
skills. Future research could examine these components in isolation or as integrated components as well as 
learning strategies, which are also not extensively researched. The use of non-instructional materials or even 
materials that the students are familiar with can also be considered in future research. A suggestion is to elicit 
students’ feedback on the types of materials they would like to view. Screening videos of the students’ choice or 
preference could be an additional form of motivation to enhance language learning and develop autonomous 
learning. Future research could also consider the number of times the materials are screened and follow up 
activities could be conducted to reinforce language learning. Next, we collected only quantitative data and 
therefore future studies could adopt a mix-method approach to include qualitative data by conducting interviews 
that would give a more lucid picture of their perception of the use of the materials. In addition, delayed 
post-test(s)/assessment tools could be administered to examine the lasting effects of viewing captioned materials. 

7. Conclusion 

To recap, the aim of this study was to examine the effects of captioned instructional video on EFL learner’s content 
comprehension, vocabulary acquisition and language proficiency. The findings clearly point to the promising 
multimedia tool of captioned instructional video as an effectives approach in enhancing EFL teaching and learning. 
Regardless of viewing the instructional video with or without captions, the gains are evident in the integrated 
components examined. However, it is also evident that the gains are significant when captions are used and the 
participants’ perception of the use of captioned instructional video was consistent with the results. They felt that it 
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enhanced their language learning, but did not affect their comprehension of the movie and that captions were not a 
form of distraction. Hence, this study supports the view that exposure to bimodal input via multi-channels results 
in better performance than exposure to only a single channel. We can conclude that the combination of visual, 
audio and print media attributes positively to language proficiency and extend Koskinen et al. (1993)’s view that 
captioned video is not only “a promising approach for improving student’s reading comprehension and 
vocabulary”, but also content comprehension and language proficiency. 

Pedagogical implications of this study are, firstly the findings of this study provides evidence for the use of 
captions/multi-channel media to enhance EFL learners’ language proficiency, which helps dispel EFL language 
instructors’ apprehension of making conscious use of captions as a teaching/learning tool. Secondly, in ESL/EFL 
contexts, where incorporating innovative multimedia technology tools into the language classrooms are restrained 
due to various factors/constraints/may be limited to the use of the basic multimedia tool of the VCR, captioned 
instructional videos can be deemed as promising media. 
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Appendices 

Appendix A: Sample of Test Questions in MET 

Sections No of questions Example of Questions Allocated Time 

a. She reads more slowly than the man does. 

b. She has a lot of material to read before she has coffee. 

Listening 25 c The man does more work than is necessary. 20 minutes 

d. The man seems to be taking a long time preparing for 
philosophy class. 

What does the passage mainly discuss? 

a. The career of Maria Edgeworth as an author of 
children’s books 

b. The development of children’s literature in the United 

Stales 35 minutes 

c. Successful publishers of children’s books in Britain and 
North America 

d. Basic differences between British and American 
literature for children 

Politicians should not 

a. earn 

Vocabulary 25 b. profit 

c. increase 

d. finance 

The “confederation school” poets of nineteenth-century 

Canada were primarily nature poets, _ a wealth of 

eulogies to Canadian rural life. 

Grammar 25 a. and producing 15 minutes 

b. who they produced 

c. producing 

d. whose production of 

Total 100 85 minutes 


from the laws they pass. 

15 minutes 


Reading 

Comprehension 


15 
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Appendix B: Samples of test items from episode 18 in CST 


Section No. of Questions Question 

Time 


Your pictures are quite_. They are so strange. 

a. Striking 


Vocabulary 4 b. Special 4 minutes 


c. Familiar 

d. Small 


Content 

Comprehension 

Have Rebecca and Diana met before? 

a. Yes 

b. No 

c. Not mentioned in the video 

4 minutes 

Total questions per ^ 
class session 


8 minutes 

Total questions at the ^ 
end of the treatment 3 


105 minutes 

3 60 content comprehension and 60 vocabulary questions 

Appendix C: Questionnaire 


Section A 

SD a D b 

NS C A d SA e 


1. My knowledge of new words increased with the help of captions. 

2. I understand the meaning of words better with the help of the actions in 
the videos. 

3.1 tend to remember the new words that I come across in the captions. 

4. When I read the words in the captions, I miss the story line. 


SD d : Strongly Disagree, D b : Disagree, NSf: Not Sure, A d : Agree, SA e : Strongly Agree 


Section B YES 

NO 

1.1 have a better understanding of the movie when I read the captions. 

2.1 learn English better with captioned videos. 
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